3 research outputs found
Solving Robust MDPs through No-Regret Dynamics
Reinforcement Learning is a powerful framework for training agents to
navigate different situations, but it is susceptible to changes in
environmental dynamics. However, solving Markov Decision Processes that are
robust to changes is difficult due to nonconvexity and size of action or state
spaces. While most works have analyzed this problem by taking different
assumptions on the problem, a general and efficient theoretical analysis is
still missing. However, we generate a simple framework for improving robustness
by solving a minimax iterative optimization problem where a policy player and
an environmental dynamics player are playing against each other. Leveraging
recent results in online nonconvex learning and techniques from improving
policy gradient methods, we yield an algorithm that maximizes the robustness of
the Value Function on the order of
where is the number of
iterations of the algorithm
Generalization Bounds for Magnitude-Based Pruning via Sparse Matrix Sketching
In this paper, we derive a novel bound on the generalization error of
Magnitude-Based pruning of overparameterized neural networks. Our work builds
on the bounds in Arora et al. [2018] where the error depends on one, the
approximation induced by pruning, and two, the number of parameters in the
pruned model, and improves upon standard norm-based generalization bounds. The
pruned estimates obtained using our new Magnitude-Based compression algorithm
are close to the unpruned functions with high probability, which improves the
first criteria. Using Sparse Matrix Sketching, the space of the pruned matrices
can be efficiently represented in the space of dense matrices of much smaller
dimensions, thereby lowering the second criterion. This leads to stronger
generalization bound than many state-of-the-art methods, thereby breaking new
ground in the algorithm development for pruning and bounding generalization
error of overparameterized models. Beyond this, we extend our results to obtain
generalization bound for Iterative Pruning [Frankle and Carbin, 2018]. We
empirically verify the success of this new method on ReLU-activated Feed
Forward Networks on the MNIST and CIFAR10 datasets
Conformalization of Sparse Generalized Linear Models
Given a sequence of observable variables , the conformal prediction method estimates a confidence set for
given that is valid for any finite sample size by merely
assuming that the joint distribution of the data is permutation invariant.
Although attractive, computing such a set is computationally infeasible in most
regression problems. Indeed, in these cases, the unknown variable can
take an infinite number of possible candidate values, and generating conformal
sets requires retraining a predictive model for each candidate. In this paper,
we focus on a sparse linear model with only a subset of variables for
prediction and use numerical continuation techniques to approximate the
solution path efficiently. The critical property we exploit is that the set of
selected variables is invariant under a small perturbation of the input data.
Therefore, it is sufficient to enumerate and refit the model only at the change
points of the set of active features and smoothly interpolate the rest of the
solution via a Predictor-Corrector mechanism. We show how our path-following
algorithm accurately approximates conformal prediction sets and illustrate its
performance using synthetic and real data examples.Comment: ICML 202